Imaging apparatus and method

ABSTRACT

An imaging apparatus includes an imaging device, a light guide mechanism, and a signal processing unit. The imaging device converts light incident on a photoelectric conversion portion of the imaging device into electric signals. The light guide mechanism, arranged adjacent to the photoelectric conversion portion of the imaging device, includes a plurality of apertures that guide light from a subject to the photoelectric conversion portion of the imaging device. The signal processing unit performs desired signal processing on the electric signals output from the imaging device on the basis of subject information units derived from the light guided onto the photoelectric conversion portion of the imaging device through the apertures.

This application is a continuation of U.S. patent application Ser. No.11/626,950, filed Jan. 25, 2007, which is a continuation-in-part of U.S.patent application Ser. No. 11/208,377, filed Aug. 19, 2005, theentireties of which are incorporated herein by reference to the extentpermitted by law. The present application claims priority to Japanesepatent application No. JP 2004-240605 filed in the Japanese PatentOffice on Aug. 20, 2004 and Japanese Patent Application No. JP2005-237139 filed in the Japanese Patent Office on Aug. 18, 2005, theentire contents both of which are incorporated by reference herein tothe extent permitted by law.

BACKGROUND OF THE INVENTION

The present invention relates to imaging apparatuses and methods, and inparticular, to an imaging apparatus using an imaging device thatconverts light from a subject into electric signals and a method ofimaging thereof.

As an imaging apparatus, a pinhole camera is well-known. In the pinholecamera, light reflected from a subject is guided onto a photosensitivematerial, such as a film, in a dark box through a hole, called apinhole, formed on one surface of the dark box, thus imaging thesubject. In the pinhole camera, slight light passing through the pinholereaches one point on the photosensitive material. Therefore, the amountof light is small. In imaging under low light conditions, particularly,in dark places, the pinhole camera is of little practical use.

A generally known imaging apparatus is disclosed by Takemura Hiroo, “CCDKamera Gijutsu Nyumon [Introduction to CCD Camera Technology]”, firstedition, Corona Publishing Co., Ltd., August 1998, pp. 2-4. The imagingapparatus includes an imaging lens 101 as shown in FIG. 24. Theapparatus has a structure in which an imaging device 102 is arranged inthe focal position of the imaging lens 101, light from a subjectcaptured by the imaging lens 101 is subjected to optical processingthrough an optical system 103 so that the imaging device 102 easilyconverts the light into electric signals, the resultant light is guidedonto a photoelectric conversion portion of the imaging device 102 sothat the light is converted into electric signals, and electric signalsobtained through the imaging device 102 are subjected to predeterminedsignal processing by a signal processing circuit 104 arranged downstreamof the imaging device 102.

This type of imaging apparatus is used solely as a camera system, e.g.,a digital still camera. Furthermore, the imaging apparatus can beincorporated into a compact portable device, such as a mobile phone.Actually, reducing the size, weight, and cost of the imaging apparatusis strongly desired in incorporating the apparatus into a mobile phone.In other words, the use of a small-sized, lightweight, and low-costimaging apparatus extremely contributes to the reduced size, weight, andcost of a compact portable device, such as a mobile phone.

SUMMARY OF THE INVENTION

Since the imaging apparatus uses the imaging lens 101, the size of theapparatus is increased by an amount corresponding to the size of theimaging lens 101, the weight thereof is increased by an amountcorresponding to the weight of the imaging lens 101, and further, thecost thereof is increased by an amount corresponding to the cost of theimaging lens 101. In addition, the amount of ambient light is reduceddue to shading of the imaging lens 101. In shooting a subject with awide dynamic range, the difference in signal charge among pixels of theimaging device 102 is large. Therefore, it is necessary to design theimaging device 102 so as to provide a wide dynamic range.

The present invention is made in consideration of the above problems. Itis desirable to provide an imaging apparatus which needs no imaging lensto achieve a reduction in size, weight, and cost of the apparatus andallows for imaging without blurring with the desired amount of light,and a method of imaging thereof.

According to an embodiment of the present invention, there is providedan imaging apparatus including an imaging device, a light guidemechanism, and a signal processing unit. The imaging device convertslight incident on a photoelectric conversion portion of the imagingdevice into electric signals. The light guide mechanism, arrangedadjacent to the photoelectric conversion portion of the imaging device,includes a plurality of apertures that guide light from a subject to thephotoelectric conversion portion of the imaging device. The signalprocessing unit performs desired signal processing on the electricsignals output from the imaging device on the basis of subjectinformation units derived from the light guided onto the photoelectricconversion portion of the imaging device through the apertures.

With the above-described structure, the light from the subject is guidedto the photoelectric conversion portion of the imaging device throughthe respective apertures, thus forming subject images corresponding tothe apertures on the photoelectric conversion portion. As for energy oflight incident on the photoelectric conversion portion of the imagingdevice, energy corresponding to the number of formed subject images,i.e., energy corresponding to the number of apertures is obtained.Therefore, the necessary amount of light can be obtained byappropriately setting the number of apertures. However, the subjectimages are formed on the photoelectric conversion portion of the imagingdevice such that the subject images are shifted relative to one anotherby the amount of shift corresponding to the pitch of the apertures. Theshifts among the subject images are corrected by the signal processingunit arranged downstream of the imaging device. Thus, a blur-free imagecan be captured with the desired amount of light.

Advantageously, the imaging apparatus according to the embodiment of thepresent invention can capture a blur-free image of a subject with thedesired amount of light using no imaging lens. Thus, a reduction insize, weight, and cost of the imaging apparatus can be achieved. Theapparatus includes no imaging lens. Accordingly, in shooting a subjectwith a wide dynamic range, the dynamic range of the imaging device canbe effectively used because the difference in signal charge among pixelsof the imaging device is small.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram illustrating the fundamental structure ofan imaging apparatus according to an embodiment of the presentinvention;

FIG. 2 is a conceptual diagram of a known optical system using a lens;

FIG. 3 is a conceptual diagram of an optical system using a plurality ofapertures according to the embodiment of the present invention;

FIG. 4 is a cross-sectional view illustrating the cross-sectional shapeof each aperture according to a first example;

FIG. 5 is a cross-sectional view illustrating the cross-sectional shapeof each aperture according to a second example;

FIG. 6 is a cross-sectional view illustrating the cross-sectional shapeof each aperture according to a third example;

FIG. 7 is a cross-sectional view illustrating the cross-sectional shapeof each aperture according to a fourth example;

FIG. 8 is a cross-sectional view illustrating the cross-sectional shapeof each aperture according to a fifth example;

FIG. 9 is a cross-sectional view illustrating the cross-sectional shapeof each aperture according to a sixth example;

FIG. 10 is a cross-sectional view illustrating the cross-sectional shapeof each aperture according to a seventh example;

FIGS. 11A and 11B are cross-sectional views illustrating other lightshielding plates;

FIGS. 12A to 12C are diagrams explaining apertures of different numbers;

FIGS. 13A to 13C are diagrams explaining apertures of different sizes;

FIGS. 14A to 14D are diagrams explaining apertures of different sizes invarious patterns of use;

FIGS. 15A to 15D are diagrams explaining apertures of differenttransmittances;

FIGS. 16A to 16D are diagrams explaining apertures of different spectraltransmission characteristics;

FIGS. 17A and 17B are diagram explaining apertures having a lens;

FIG. 18 shows an example of the transmittance distribution of anaperture;

FIG. 19 shows an example of the transmittance distribution of anaperture;

FIG. 20 shows an example of the transmittance distribution of anaperture;

FIG. 21 shows examples of transmittance distribution functions;

FIG. 22 is a diagram showing the relationship between a subject and theimaging apparatus using a one-dimensional model;

FIG. 23 is a diagram illustrating the relationship between the subjectand the imaging apparatus using a two-dimensional model; and

FIG. 24 is a schematic diagram illustrating the fundamental structure ofan imaging apparatus using an imaging lens.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

An embodiment of the present invention will now be described in detailbelow with reference to the drawings.

FIG. 1 is a schematic diagram illustrating the fundamental structure ofan imaging apparatus according to an embodiment of the presentinvention. Referring to FIG. 1, an apparatus body 11 has an opening atone principal plane thereof. A package 12 having, for example, arectangular shape is arranged at the opening. The package 12 has anopening at one principal plane thereof such that the opening is adjacentto the opening of the body 11. A light shielding plate 13 is arranged atthe opening of the package 12, thus forming a dark box.

An imaging device 14 is arranged in the package 12. The imaging device14 includes a solid imaging device, e.g., a charge-transfer imagingdevice, such as a charge coupled device (CCD) imager, or anXY-addressable imaging device, such as a metal oxide semiconductor (MOS)imager. The solid imaging device converts incoming light into anelectric signal based on the amount of light every pixel.

In the present embodiment, for example, the light shielding plate 13includes, for example, a flat plate. The light shielding plate 13 has aplurality of apertures 15 whose number is the same as that of pixels ofthe imaging device 14. The apertures 15, called pinholes, each have avery small diameter.

The light shielding plate 13 and the apertures 15 arranged in the lightshielding plate 13 constitute an aperture sheet that functions as alight guide mechanism that guides light from a subject 17 to aphotoelectric conversion section of the imaging device 14. Accordingly,light beams from the subject 17 pass through the respective apertures15, so that subject images as much as the apertures 15 are formed assubject information units on an imaging surface of the imaging device 14by the pinhole effect.

The imaging device 14 converts each subject image (subject informationunit) formed on the imaging surface into an electric signal every pixeland outputs the electric signals. A signal processing circuit 16 isarranged downstream of the imaging device 14. The signal processingcircuit 16 performs desired signal processing on the electric signalsoutput from the imaging device 14 on the basis of the subjectinformation units. Specifically, the signal processing includescorrection of the difference among the subject information units,subject image detection, subject motion detection, and subjectrecognition. The signal processing will be described in detail below.

The imaging principle of the imaging apparatus with the above structureaccording to the present embodiment will now be described.

Light from the subject 17 is guided into the package 12, serving as thedark box, through the apertures 15 in the light shielding plate 13. Atthat time, each aperture 15 forms a subject image on the imaging surfaceof the imaging device 14 by the pinhole effect. In this instance, thesubject images corresponding to the respective apertures 15 are formedon the imaging surface of the imaging device 14. Consequently, theamount of light corresponding to the number of formed subject images,i.e., the number of apertures 15 is obtained on the whole imagingsurface.

An optical system using the apertures 15, constituting the light guidemechanism, according to the present embodiment will be compared with aknown optical system using a lens with respect to the energy(brightness) of light incident on the imaging device with reference toFIGS. 2 and 3. FIG. 2 is a conceptual diagram of the known opticalsystem using a lens. FIG. 3 is a conceptual diagram of the opticalsystem using the apertures 15 according to the present embodiment of thepresent invention.

Regarding the F-number (focal length/diaphragm aperture diameter) of alens in an imaging apparatus used in a digital still camera or a mobilephone, the F-number is about 2.8. On the other hand, the F-number in theimaging apparatus according to the present embodiment is obtained by thefollowing expression on the assumption that the diameter p of eachaperture 15 is 3 μm and the distance L between the light shielding plate13 and the imaging surface of the imaging device 14 is 3 mm.

Fa=L/p=3000 μm/3 μm=1000

Therefore, the ratio of the F-number of the lens, i.e., F=2.8 to thataccording to the present embodiment, i.e., in the use of the apertures15 instead of a lens is obtained by the following expression:

F/Fa=2.8/1000

In this case, the ratio of energy of light incident on the imagingdevice 14 in the case using the lens and that in the case of using theapertures is as follows:

(F/Fa)²=0.00000784

The reciprocal thereof is 127,551.

For example, when a device of which the pixel size is 3 μm, the numberof pixels in the horizontal direction is 1,000, the number of pixels inthe vertical direction is 1,000, and the total number of pixels is1,000,000 is used as the imaging device 14, the number of apertures 15is 127, 551 on the condition that the energy of light incident on theimaging device 14 is equivalent to that in the optical system using thelens with F=2.8.

As will be obviously understood from a result of the above-describedcomparison, about 100,000 apertures 15 are formed in the light shieldingplate 13, thus obtaining the energy (brightness) of light equivalent tothat of a known imaging apparatus used in a digital still camera or amobile phone. According to the present embodiment, therefore, theapertures 15 whose number corresponds to the total number of pixels ofthe imaging device 14 are formed, thus providing enough energy as theenergy of light incident on the imaging device 14, the energy beinghigher than that in an imaging apparatus used in a digital still cameraor a mobile phone.

The number of apertures 15 is described above. Regarding the size of anaperture formation region where the apertures 15 are formed in the lightshielding plate 13, in order to permit the beams of light to be incidenton pixels in the periphery of an effective pixel region (includingpixels actually used for imaging information) of the imaging device 14so as to provide energy equivalent to that of pixels in the center ofthe effective pixel region, the aperture formation region is set largerthan the effective pixel region of the imaging device 14. Preferably,the area ratio of the aperture formation region to the effective pixelregion is, for example, 9:1.

The size of the aperture formation region in the light shielding plate13 relative to the effective pixel region of the imaging device 14 alsodepends on the relationship between the angle of view of the imagingdevice 14 and the distance L between the light shielding plate 13 andthe imaging surface of the imaging device 14. For instance, assumingthat the effective pixel region is 3 mm (in the horizontal direction)×3mm (in the vertical direction) and the distance L is 3 mm, when theangle of view is set to 90 degrees, the area of the aperture formationregion in the light shielding plate 13 is nine times as large as theeffective pixel region. Thus, the energy of light incident on all of thepixels in the effective pixel region can be substantially uniformed.

As described above, subject information units, i.e., subject images areformed on the imaging surface of the imaging device 14 through theapertures 15, so that the energy of light corresponding to the number ofapertures 15 can be obtained on the entire imaging surface. Therefore,the necessary amount of light can be obtained by appropriately settingthe number of apertures 15. However, the subject images are formed onthe imaging surface such that the formed images are shifted relative toone another by the amount of shift corresponding to the pitch of theapertures 15, i.e., the subject information units differ from oneanother by the amount of shift. The shifts among the subject images(i.e., the differences among subject information units) formed on theimaging surface are corrected by signal processing performed by thesignal processing circuit 16. The details of the signal processing willbe described below.

Shape of Aperture

In this instance, the shape of each aperture 15 will now be described.The apertures 15 with various shapes are available. Some examples of theshapes of the apertures 15 will be described below.

First Example

FIG. 4 is a cross-sectional view illustrating the cross-sectional shapeof each aperture 15A according to a first example. In the first example,the cross-sectional area of each aperture 15A is uniform in thedirection from the subject 17 to the imaging device 14, i.e., in thedirection from the surface of the light shielding plate 13 close to thesubject 17 to the surface thereof close to the imaging device 14. Inaddition, the axis O of each aperture 15A is orthogonal to the surfaceof the light shielding plate 13. In other words, the apertures 15 havethe same shape.

Second Example

FIG. 5 is a cross-sectional view illustrating the cross-sectional shapeof each aperture 15B according to a second example. In the secondexample, the cross-sectional area of each aperture 15B is uniform in thedirection from the surface of the light shielding plate 13 close to thesubject 17 to the surface thereof close to the imaging device 14 suchthat the axes O of the respective apertures 15B are converged to theimaging device 14. In other words, the inclination of the axis O of eachaperture 15B depends on the position of the aperture 15B. Therefore, theshape of the aperture 15B depends on the position thereof.

Third Example

FIG. 6 is a cross-sectional view illustrating the cross-sectional shapeof each aperture 15C according to a third example. In the third example,the cross-sectional area of each aperture 15C is uniform in thedirection from the surface of the light shielding plate 13 close to thesubject 17 to the surface thereof close to the imaging device 14 suchthat the axes O of the respective apertures 15C are diverged toward theimaging device 14. In other words, the inclination of the axis O of eachaperture 15C depends on the position of the aperture 15C. Therefore, theshape of the aperture 15C depends on the position thereof.

Fourth Example

FIG. 7 is a cross-sectional view illustrating the cross-sectional shapeof each aperture 15D according to a fourth example. In the fourthexample, the cross-sectional area of each aperture 15D varies in thedirection from the surface of the light shielding plate 13 close to thesubject 17 to the surface thereof close to the imaging device 14.Specifically, the cross-sectional thereof gradually increases in thisdirection. The axes O of the respective apertures 15D are orthogonal tothe surface of the light shielding plate 13. In other words, theapertures 15D have the same shape.

Fifth Example

FIG. 8 is a cross-sectional view illustrating the cross-sectional shapeof each aperture 15E according to a fifth example. In the fifth example,the cross-sectional area of each aperture 15E gradually increases in thedirection from the surface of the light shielding plate 13 close to thesubject 17 to the surface thereof close to the imaging device 14 and theaxes O of the respective apertures 15E are orthogonal to the surface ofthe light shielding plate 13 in the same way as the apertures 15Daccording to the fourth example. However, the opening area of eachaperture 15E at the surface of the light shielding plate 13 close to theimaging device 14 is larger than that of the aperture 15D according tothe fourth example.

Sixth Example

FIG. 9 is a cross-sectional view illustrating the cross-sectional shapeof each aperture 15F according to a sixth example. In the sixth example,the direction of spreading (hereinafter, orientation) of each aperture15F is opposite to that of the aperture 15D according to the fourthexample. In other words, the cross-sectional area of each aperture 15Fgradually decreases in the direction from the surface of the lightshielding plate 13 close to the subject 17 to the surface thereof closeto the imaging device 14 and the axes O of the respective apertures 15Fare orthogonal to the surface of the light shielding plate 13.

Seventh Example

FIG. 10 is a cross-sectional view illustrating the cross-sectional shapeof each aperture 15G according to a seventh example. In the seventhexample, the orientation of each aperture 15G is opposite to that of theaperture 15E according to the fifth example. In other words, thecross-sectional area of each aperture 15G gradually decreases in thedirection from the surface of the light shielding plate 13 close to thesubject 17 to the surface thereof close to the imaging device 14 and theaxis O of the respective apertures 15G are orthogonal to the surface ofthe light shielding plate 13 in the same way as the apertures 15Faccording to the sixth example. However, the opening area of eachaperture 15G at the surface of the light shielding plate 13 close to thesubject 17 is larger than that of the aperture 15F according to thesixth example.

The seven examples of the shape of the aperture 15 have been described.The shape of the aperture 15 is not limited to those examples. Any ofthe apertures 15A to 15G is appropriately selected as an aperture havingthe shape suitable for imaging conditions. Particularly, the use of theapertures each having the cross-sectional area varying in the directionfrom the surface of the light shielding plate 13 close to the subject 17to the surface thereof close to the imaging device 14, i.e., any of theapertures 15D to 15G according to the fourth to seventh examples reducesthe loss of light incident on the imaging device 14. Advantageously, thesensitivity of the present imaging apparatus can be increased.

In the fourth and fifth examples, the apertures 15D and 15E each have alarge opening that is close to the imaging device 14. The apertures ofthis type will be termed “apertures of a first orientation”. In thesixth and seventh examples, the apertures 15F and 15G each have a largeopening that is close to the subject 17. The apertures of this type willbe termed “apertures of a second orientation”. In each of the fourth toseventh examples, all of the apertures have the same orientation. Theapertures may have different orientations, i.e., the apertures of thefirst and second orientations may be arranged.

The arrangement of apertures of different orientations, i.e., the firstand second orientations increases the intensity of light incident on theperiphery of the imaging device 14, enabling an improvement in shading.

In the above-described first to seventh examples of the apertures 15,the apertures 15 are arranged in the flat light shielding plate 13. Thelight shielding plate 13 in which the apertures 15 are arranged is notlimited to a flat plate. For example, a curved plate can be used.

Referring to FIG. 11A, a convex light shielding plate 13A whichprotrudes toward the subject 17 can be used. Any type of the apertures15A to 15G according to the above-described first to seventh examplesmay be arranged in the light shielding plate 13A. Referring to FIG. 11B,a concave light shielding plate 13B which protrudes toward the imagingdevice 14 can be used. Any type of the apertures 15A to 15G according tothe above-described first to seventh examples may be arranged in thelight shielding plate 13B.

The use of the curved plate, serving as the light shielding plate 13,increases the intensity of light incident on the periphery of theimaging device 14, enabling an improvement in shading.

Modifications

The foregoing embodiment has been described on the assumption that theapertures 15 are arranged at the same pitch. However, it is unnecessaryto arrange the apertures 15 at the same pitch. The apertures 15 may bearranged at different pitches.

In the foregoing embodiment, the number of apertures 15 is the same asthe number of pixels of the imaging device 14. However, it isunnecessary to arrange the apertures 15 as much as the pixels of theimaging device 14. The number of apertures 15 may be arbitrarily set.Some examples are shown in FIGS. 12A to 12C. FIG. 12A illustratesarrangement of twelve apertures. FIG. 12B illustrates arrangement offour apertures. FIG. 12C illustrates arrangement of two apertures.

The size of each aperture 15 is not fixed. FIGS. 13A to 13C illustratesome examples. FIG. 13A illustrates arrangement of relatively middlesized apertures. FIG. 13B illustrates arrangement of large apertures.FIG. 13C illustrates arrangement of small apertures. The apertures 15may have any size. Furthermore, different sized apertures, for example,three different sized apertures may be arranged as shown in FIG. 14A.Various patterns of use of those apertures are available. Referring toFIG. 14B, the middle sized apertures may be used such that the small andlarge apertures are blocked. Referring to FIG. 14C, the small aperturesmay be used such that the large and middle sized apertures are blocked.Referring to FIG. 14D, the large apertures may be used such that thesmall and middle sized apertures are blocked.

The transmittance of each aperture 15 is not limited to a predeterminedtransmittance. The transmittance thereof can be arbitrarily set as shownin FIGS. 15A to 15C. FIG. 15A illustrates arrangement of apertureshaving a maximum transmittance. FIG. 15B illustrates arrangement ofapertures having a medium transmittance. FIG. 15C illustratesarrangement of apertures having a minimum transmittance. Furthermore,apertures of at least two different transmittances may be arranged. Asshown in FIG. 15D, apertures of three different transmittances, i.e.,the maximum, medium, and minimum transmittances may be arranged.

Since the apertures of at least two different transmittances arearranged as described above, subject images can be formed on the imagingsurface of the imaging device 14 so as to provide a light intensitydistribution based on the respective transmittances. It is assumed thata minimum transmittance of 0% is achieved by completely blocking therelevant aperture. When a shutter is arranged to each aperture 15 inorder to achieve the minimum transmittance, a shutter of a camera is notneeded. Advantageously, the size and weight of the present imagingapparatus can be reduced.

The spectral transmission characteristics of the apertures 15 will nowbe described with reference to FIGS. 16A to 16D. When the apertures 15are completely transparent holes, the spectral transmissioncharacteristics cover the full wavelength range. On the other hand, thespectral transmission characteristics in a specific wavelength range canbe provided. FIG. 16B illustrates arrangement of apertures having thespectral transmission characteristics in the red wavelength range. FIG.16C illustrates arrangement of apertures having the spectraltransmission characteristics in the green wavelength range. FIG. 16Dillustrates arrangement of apertures having the spectral transmissioncharacteristics in the blue wavelength range. Since the apertures 15have the spectral transmission characteristics, a prism for a 3-CCDimaging system is not needed. Advantageously, the size and weight of thepresent imaging apparatus can be reduced.

Each aperture 15 may have a lens. Attaching a lens to each aperture 15can permit the solid angle, intensity, and angle of light incident onthe imaging device 14 to have flexibility derived from the design of thelens. Regarding attachment of lenses to the apertures 15, variouspatterns are available. FIG. 17A illustrates arrangement of apertureseach having a lens. A combination of at least one aperture having a lensand at least one aperture having no lens may be used. FIG. 17Billustrates a combination of a large aperture having a lens andapertures having no lens.

Particularly, in the use of arrangement of at least one aperture havinga lens and at least one aperture having no lens, when the ambient lightis dark, light passing through the lens which has a small F-number(bright) is used. When the ambient light is bright, light passingthrough the lens is controlled and light passing through the aperturehaving no lens is used. As for the amount of light, a wide variablerange of F-number from 1.4 to about 1000 can be realized in practicaluse.

Even when the intensity of light passing through the lens is so highthat the pixels of the imaging device 14 are partially saturated, signalprocessing is performed on the basis of a signal output from the imagingdevice 14, the signal being based on light passing through the aperturehaving no lens. Thus, an image having a wide dynamic range can beobtained. Furthermore, various processes, such as electronic zooming,stereoscopic imaging, high resolution processing, wide dynamic rangeprocessing, color characteristics processing, demosaic processing, lownoise processing, and high sensitivity processing, can be performed by asingle camera system.

Signal Processing Circuit

The signal processing circuit 16 for correcting the differences amongsubject information units (subject images) formed on the imaging surfaceof the imaging device 14 through the apertures 15 will now be describedbelow.

The signal processing circuit 16 performs arithmetic processing using acoefficient determined on the basis of the intensity distribution oflight incident on the imaging device 14 to obtain subject informationunits. The intensity distribution of light incident on the imagingdevice 14 is determined on the basis of the pitch of the apertures 15,the size of each aperture 15, the shape thereof, the cross-sectionalshape thereof, the orientation thereof, the position thereof, thetransmittance distribution of each aperture 15, the distance betweeneach aperture 15 and the imaging device 14, the distance between eachaperture 15 and the subject 17, the diffraction of light through eachaperture 15, and the interference of light through the apertures 15.

Particularly, the transmittance distributions of the apertures 15 arechanged, thus preventing the influence of diffraction of light passingthrough the apertures 15, specifically, a decrease in resolution due tothe diffraction. The same advantages can be obtained in not only the useof the plurality of apertures 15 but also the use of a single aperture15.

FIGS. 18 to 20 illustrate examples of the transmittance distribution ofthe aperture 15. FIG. 18 illustrates a uniform apodization function.FIG. 19 shows a Blackman apodization function. FIG. 20 illustrates aBartlett apodization function.

The term “apodization” means a method of improving the imagingproperties of an optical system. Apodization is achieved by allowing theapertures 15 to have the transmittance distribution in order to reducethe intensity of diffraction rings surrounding the Airy disk, serving asan image with no aberration created by diffracting light from a pinpointlight source through the aperture.

FIG. 21 illustrates examples of other functions with respect to thetransmittance distribution. The apodization functions B_(A)(x),B_(I)(k), Hm_(A)(x), Hm_(I)(k), Hn_(A)(x), Hn_(I)(k), and W_(I)(k) areexpressed by Expression 1.

$\begin{matrix}{\begin{matrix}{{B_{A}(I)} = {0.42 + {0.5{\cos \left( \frac{\pi \; x}{a} \right)}} + {0.08{\cos \left( \frac{{2\pi \; x}\;}{a} \right)}}}} \\{{B_{I}(k)} = \frac{{a\left( {0.84 - {0.36a^{2}k^{2}}} \right)}\sin \; {c\left( {2\pi \; {ak}} \right)}}{\left( {1 - {a^{2}k^{2}}} \right)\left( {1 - {4a^{2}k^{2}}} \right)}}\end{matrix}{{{Hm}_{A}(I)} = {0.54 + {0.46{\cos \left( \frac{\pi \; x}{a} \right)}}}}{{{Hm}_{I}(k)} = \frac{{a\left( {1.08 - {0.64a^{2}k^{2}}} \right)}\sin \; {c\left( {2\pi \; {ak}} \right)}}{1 - {4a^{2}k^{2}}}}\begin{matrix}{{{Hn}_{A}(I)} = {\cos^{2}\left( \frac{\pi \; x}{2a} \right)}} \\{= {\frac{1}{2}\left\lbrack {1 + {\cos \left( \frac{\pi \; x}{a} \right)}} \right\rbrack}}\end{matrix}\begin{matrix}{{{Hn}_{I}(k)} = \frac{a\; \sin \; {c\left( {2\pi \; {ak}} \right)}}{1 - {4a^{2}k^{2}}}} \\{= {a\left\lbrack {{\sin \; {c\left( {2\pi \; {ka}} \right)}} + {\frac{1}{2}\sin \; {c\left( {{2\pi \; {ka}} - \pi} \right)}} + {\frac{1}{2}\sin \; {c\left( {{2\pi \; {ka}} + \pi} \right)}}} \right\rbrack}}\end{matrix}\begin{matrix}{{W_{I}(k)} = {a\; 2\sqrt{2\pi}\frac{J_{3/2}\left( {2\pi \; {ka}} \right)}{\left( {2\pi \; {ka}} \right)^{3/2}}}} \\{= {a{\frac{{\sin \left( {2\pi \; {ka}} \right)} - {2\pi \; {ak}\; {\cos \left( {2\pi \; {ak}} \right)}}}{2a^{3}k^{3}\pi^{3}}.}}}\end{matrix}} & (1)\end{matrix}$

For the sake of easy understanding of signal processing by the signalprocessing circuit 16, a one-dimensional model shown in FIG. 22 will bedescribed as an example of the imaging device 14. In the one-dimensionalmodel, pixels are arranged one-dimensionally. Referring to FIG. 22,reference symbol L denotes the distance between the light shieldingplate 13 and the imaging surface of the imaging device 14, P denotes thepitch of the apertures 15, p denotes the diameter of each aperture 15, Adenotes the aperture ratio (=p/P), and m indicates a coefficient. Inthis case, it is assumed that the pitch P is equal to the pitch betweenpixels.

Information Si supplied to a pixel Si (i denotes a pixel number) of theimaging device 14 is expressed by Expression 2:

$\begin{matrix}{\mspace{79mu} {{S_{i} = {{\sum\limits_{j = {- N}}^{N}{k_{ij}{B_{j}\begin{pmatrix}S_{- N} \\S_{- {({N - 1})}} \\S_{i} \\S_{N}\end{pmatrix}}}} = {\begin{pmatrix}k_{{- N} - N} & k_{{- N} - {({N - 1})}} & k_{- {Ni}} & k_{- {NN}} \\k_{{- {({N - 1})}} - N} & k_{{- {({N - 1})}} - {({N - 1})}} & k_{{- {({N - 1})}}j} & k_{{- {({N - 1})}}N} \\k_{i - N} & k_{i - {({N - 1})}} & k_{ij} & k_{iN} \\k_{N - N} & k_{N - {({N - 1})}} & k_{Nj} & k_{NN}\end{pmatrix}\begin{pmatrix}B_{- N} \\B_{- {({N - 1})}} \\B_{j} \\B_{N}\end{pmatrix}}}}\mspace{79mu} {{i = {- N}},{- \left( {N - 1} \right)},{\ldots \mspace{14mu} - 1},0,1,2,\ldots \mspace{14mu},N}\mspace{79mu} {{j = {- N}},{- \left( {N - 1} \right)},{\ldots \mspace{14mu} - 1},0,1,2,\ldots \mspace{14mu},N}\mspace{79mu} {k_{ij} = {{Acos}^{2}\theta_{ij}}}}} & (2)\end{matrix}$

where k_(ij) is a coefficient defined by the pitch P of the apertures 15and the diameter p of the aperture 15.

Information (in this instance, light including visible light orelectromagnetic radiation, such as near infrared radiation, infraredradiation, or ultraviolet radiation) Bj from the subject 17 is obtainedby arithmetic processing. If the information Si relates to only thebrightness of the subject 17, brightness information can be reproducedas the information Bj by arithmetic processing. When the information Sirelates to the color of the subject 17, color information can bereproduced as the information Bj by arithmetic processing.

If there is brightness information alone, the brightness information Bjof the subject 17 can be calculated by arithmetic processing based onExpression 3. To reproduce the color of the subject 17, informationunits related to several kinds of colors, such as red, blue, and green,are obtained as output signals of pixels. Thus, color information of thesubject 17 can be similarly obtained by arithmetic processing based onExpression 3.

$\begin{matrix}{\mspace{79mu} {{B_{j} = {f\left( {S_{i},k_{ij}} \right)}}{\begin{pmatrix}B_{- N} \\B_{- {({N - 1})}} \\B_{j} \\B_{N}\end{pmatrix} = {\begin{pmatrix}k_{{- N} - N} & k_{{- N} - {({N - 1})}} & k_{- {Nj}} & k_{- {NN}} \\k_{{- {({N - 1})}} - N} & k_{{- {({N - 1})}} - {({N - 1})}} & k_{{- {({N - 1})}}j} & k_{{- {({N - 1})}}N} \\k_{i - N} & k_{i - {({N - 1})}} & k_{ij} & k_{iN} \\k_{N - N} & k_{N - {({N - 1})}} & k_{Nj} & k_{NN}\end{pmatrix}^{- 1}\begin{pmatrix}S_{- N} \\S_{- {({N - 1})}} \\S_{i} \\S_{N}\end{pmatrix}}}\mspace{20mu} {{i = {- N}},{- \left( {N - 1} \right)},{\ldots \mspace{14mu} - 1},0,1,2,\ldots \mspace{14mu},N}\mspace{20mu} {{j = {- N}},{- \left( {N - 1} \right)},{\ldots \mspace{14mu} - 1},0,1,2,\ldots \mspace{14mu},N}\mspace{20mu} {k_{ij} = {A\; \cos^{2}\theta_{ij}}}}} & (3)\end{matrix}$

As approaches to obtaining color information as a pixel output signal,all of known color separating methods are available. For example, amethod for achieving color separation through a prism and imaging usinga 3-chip (3-CCD) imaging system, a method using an on-chip color filteron an imaging device, a method of arranging color filters in theapertures 15, and a method for achieving color separation through apixel portion of an imaging device can be used.

As described above, the information Bj, obtained by arithmeticprocessing through the signal processing circuit 16, as brightnessinformation or color information of the subject 17 is shown on adisplay, thus reproducing an image of the subject 17 as aone-dimensional image.

For the sake of easy understanding, the one-dimensional model has beendescribed above as an example. FIG. 23 shows a two-dimensional model. Inthe case of the two-dimensional model, Expression 4 and Expression 5correspond to Expression 2 and Expression 3 for the one-dimensionalmodel, respectively. The signal processing circuit 16 performsarithmetic processing based on those expressions to obtain informationBjk as brightness information or color information of the subject 17.When the information Bjk is shown on a display, an image of the subject17 can be reproduced as a two-dimensional image.

$\begin{matrix}{\mspace{79mu} {{S_{hi} = {{\sum\limits_{{j = {- N}},{k = {- M}}}^{N,M}{k_{hijk}{B_{jk}\begin{pmatrix}S_{{- N},i} \\S_{{- {({N - 1})}}j} \\S_{h,i} \\S_{N,i}\end{pmatrix}}}} = {\sum\limits_{k = {- M}}^{M}\left( {\begin{pmatrix}k_{{- N},i,{- N},k} & k_{{- N},{i - {({N - 1})}},k} & k_{{- N},i,j,k} & k_{{- N},i,N,k} \\k_{{{{- {({N - 1})}}i} - N},k} & k_{{- {({N - 1})}},i,{- {({N - 1})}},k} & k_{{- {({N - 1})}},i,j,k} & k_{{- {({N - 1})}},i,N,k} \\k_{h,i,{- N},k} & k_{h,i,{- {({N - 1})}},k} & k_{h,i,j,k} & k_{h,i,N,k} \\k_{N,i,{- N},k} & k_{N,i,{- {({N - 1})}},k} & k_{N,i,j,k} & k_{N,i,N,k}\end{pmatrix}\begin{pmatrix}B_{{- N},k} \\B_{{- {({N - 1})}},k} \\B_{i,k} \\B_{N,k}\end{pmatrix}} \right)}}}\mspace{20mu} {{h = {- N}},{- \left( {N - 1} \right)},{\ldots \mspace{14mu} - 1},0,1,2,\ldots \mspace{14mu},N}\mspace{20mu} {{i = {- M}},{- \left( {M - 1} \right)},{\ldots \mspace{14mu} - 1},0,1,2,\ldots \mspace{14mu},M}\mspace{20mu} {{j = {- N}},{- \left( {N - 1} \right)},{\ldots \mspace{14mu} - 1},0,1,2,\ldots \mspace{14mu},N}\mspace{20mu} {{k = {- M}},{- \left( {M - 1} \right)},{\ldots \mspace{14mu} - 1},0,1,2,\ldots \mspace{14mu},M}\mspace{20mu} {k_{hijk} = {A\; \cos^{2}\theta_{hijk}}}}\mspace{14mu}} & (4)\end{matrix}$

In Expression 4, k_(hijk) is a coefficient determined by the pitch P ofthe apertures 15 and the diameter p of the aperture 15.

B _(jk)=ƒ(S _(hi) , k _(hijk))

where f is the inverse function of the following:

$\begin{matrix}{{{\begin{pmatrix}S_{{- N},i} \\S_{{- {({N - 1})}},i} \\S_{h,i} \\S_{N,i}\end{pmatrix} = {\sum\limits_{k = {- M}}^{M}\left( {\left( \begin{matrix}k_{{- N},i,{- N},k} & k_{{- N},{i - {({N - 1})}},k} & k_{{- N},i,j,k} & k_{{- N},i,N,k} \\k_{{- {({N - 1})}},i,{- N},k} & k_{{- {({N - 1})}},i,{- {({N - 1})}},k} & k_{{- {({N - 1})}},i,j,k} & k_{{- {({N - 1})}},i,N,k} \\k_{h,i,{- N},k} & k_{h,i,{- {({N - 1})}},k} & k_{h,i,j,k} & k_{h,i,N,k} \\k_{N,i,{- N},k} & k_{N,i,{- {({N - 1})}},k} & k_{N,i,j,k} & k_{N,i,N,k}\end{matrix} \right)\begin{pmatrix}B_{{- N},k} \\B_{{- {({N - 1})}},k} \\B_{i,k} \\B_{N,k}\end{pmatrix}} \right)}}\mspace{20mu} {{h = {- N}},{- \left( {N - 1} \right)},{\ldots \mspace{14mu} - 1},0,1,2,\ldots \mspace{14mu},N}\mspace{20mu} {{i = {- M}},{- \left( {M - 1} \right)},{\ldots \mspace{14mu} - 1},0,1,2,\ldots \mspace{14mu},M}\mspace{20mu} {{j = {- N}},{- \left( {N - 1} \right)},{\ldots \mspace{14mu} - 1},0,1,2,\ldots \mspace{14mu},N}\mspace{20mu} {{k = {- M}},{- \left( {M - 1} \right)},{\ldots \mspace{14mu} - 1},0,1,2,\ldots \mspace{14mu},M}\mspace{20mu} {k_{hijk} = {A\; \cos^{2}\theta_{hijk}}}}\mspace{14mu}} & (5)\end{matrix}$

As described above, light from the subject 17 is guided to thephotoelectric conversion portion of the imaging device 14 through therespective apertures 15 arranged in the light shielding plate 13disposed adjacent to the photoelectric conversion portion of the imagingdevice 14. Electric signals, obtained by photoelectrically convertingthe light from the subject 17 through the imaging device 14, aresubjected to desired signal processing on the basis of subjectinformation units corresponding to the respective apertures 15,specifically, signal processing for correction of the differences amongthe subject information units. Thus, a blur-free image of the subjectcan be captured with the desired amount of light without using animaging lens. This leads to a reduction in size, weight, and cost of theimaging apparatus.

Since any imaging lens is not used, the difference in signal chargeamong pixels of the imaging device 14 is small when a subject with awide dynamic range is captured. Thus, the dynamic range of the imagingdevice 14 can be effectively utilized. Furthermore, since the presentimaging apparatus with no lens has an appropriate structure, the amountof light incident on pixels in the vicinity of the effective pixelregion can be prevented from decreasing.

Application of the above-described signal processing circuit 16 is notlimited to the imaging apparatus having the apertures 15. The signalprocessing circuit 16 can be applied to an imaging apparatus having asingle aperture 15.

In the foregoing embodiment, signal processing by the signal processingcircuit 16 relates to the correction of the differences among subjectimages (subject information units) formed on the imaging surface of theimaging device 14 through the respective apertures 15. In addition tothe correction of the differences to obtain an image of a subject, thesignal processing may include various processes, e.g., subject motiondetection and subject recognition.

The imaging apparatus according to the foregoing embodiment can be usedsolely as a general camera system, such as a digital still camera. Sincethe size, weight, and cost of the apparatus can be reduced, theapparatus can be incorporated as a camera module into a compact portabledevice, such as a mobile phone. This greatly contributes to the reducedsize, weight, and cost of the compact portable device.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. An imaging apparatus comprising: a photoelectric conversion portionwith a plurality of pixel units which convert light incident on theplurality of pixel units into a plurality of electrical signals; a lightguide unit adjacent the photoelectric conversion portion and including aplurality of apertures through which light is guided to the plurality ofpixel units of the photoelectric conversion portion; and a signalprocessing unit which processes the plurality of electric signals,wherein, one pixel unit generates its electrical signal from lightguided onto the photoelectric conversion unit through more than oneaperture.
 2. The imaging apparatus of claim 1, wherein the apertures areall of the same size.
 3. The imaging apparatus of claim 1, wherein atleast two of the apertures are of different sizes.
 4. The imagingapparatus of claim 1, wherein the apertures are all of the same shape.5. The imaging apparatus of claim 1, wherein at least two of theapertures are of different shapes.
 6. The imaging device of claim 1,wherein all of the apertures have an axis along the direction the lightis guided therethrough, and all the axis have the same orientation withrespect to the light conversion portion.
 7. The imaging device of claim1, wherein all of the apertures have an axis along the direction thelight is guided therethrough, and at least two of the axis havedifferent orientations with respect to the light conversion portion. 8.The imaging device of claim 1, wherein all of the apertures have thesame light transmittance characteristics.
 9. The imaging device of claim1, wherein at least two apertures have different light transmittancecharacteristics.
 10. The imaging device of claim 1, wherein all of theapertures have the same spectral transmission characteristics.
 11. Theimaging device of claim 1, wherein at least two apertures have differentspectral transmission characteristics.
 12. The imaging device of claim1, comprising at least one lens respectively associated with one of theapertures.
 13. The imaging device of claim 1, comprising at least onelens respectively associated with one of the apertures, and at least oneaperture without a lens associated therewith.
 14. The imaging device ofclaim 1, wherein the number of apertures is the same as the number ofpixel units.
 15. The imaging device of claim 1, wherein the number ofapertures and the number of pixel units are different.
 16. The imagingdevice of claim 1, wherein the light guide unit comprises a lightshielding plate through which the apertures extend, the light shieldingplate being planar.
 17. The imaging device of claim 1, wherein the lightguide unit comprises a light shielding plate through which the aperturesextend, the light shielding plate being curved in cross-section.
 18. Theimaging device of claim 17, wherein the light shielding plate is concaveand protrudes toward the photoelectric conversion portion.
 19. Theimaging device of claim 17, wherein the light shielding plate is convexand protrudes away from the photoelectric conversion portion.
 20. Theimaging device of claim 1, wherein the apertures have axis along thedirection of travel of light therethrough, and the axis are orientedsuch that they converge toward the photoelectric conversion portion. 21.The imaging device of claim 1, wherein the apertures have axis along thedirection of travel of light therethrough, and the axis are orientedsuch that they diverge away from the photoelectric conversion portion.22. The imaging device of claim 1, wherein at least one aperture a firstopening through which the light enters the aperture and a second openingthrough which the light exits the aperture, and the first and secondopenings are of different sizes.
 23. The imaging device of claim 22,wherein the first opening is larger in diameter than the second opening.24. The imaging device of claim 22, wherein the second opening is largein diameter than the first opening.